Skip to content

Add recall quality evidence dashboard#33

Merged
TerminallyLazy merged 12 commits into
mainfrom
codex/recall-quality-dashboard
Jul 9, 2026
Merged

Add recall quality evidence dashboard#33
TerminallyLazy merged 12 commits into
mainfrom
codex/recall-quality-dashboard

Conversation

@TerminallyLazy

Copy link
Copy Markdown
Owner

Summary

  • add deterministic recall-quality diagnostics using generated fixture memories in an isolated in-memory SQLite store
  • write recall-quality evidence into target/tree-ring-certification/recall-quality/default-fixture-v1.json and merge it into evidence-index.json
  • surface recall-quality status, counts, latency, query ids, and returned ids in the TUI /evidence view without rendering memory summaries/details
  • wire recall-quality validation into scripts/certify-tree-ring.sh with structured JSON assertions and update README docs

Verification

  • cargo fmt --check
  • git diff --check
  • cargo test --locked (119 CLI, 63 core, 34 SQLite tests, doc tests)
  • cargo clippy --locked --all-targets
  • sh scripts/certify-tree-ring.sh
    • release binary: 6,352,528 bytes
    • project install: 6,272 KB
    • global install: 6,228 KB
    • CLI import: 10,000 memories in 5s (2,000/s)
    • evidence index: overall_status pass, missing []
    • recall quality: status pass, query_count 4, fail_count 0, sensitive_fixture_count 1, private_payloads_used false

Review

Final subagent review: ready to merge after structured certification validation fix.

@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

An error occurred during the review process. Please try again later.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/recall-quality-dashboard

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements Phase 3 of the evidence spine by introducing deterministic recall-quality diagnostics to evaluate recall speed and relevance using safe fixture memories. It adds a new recall-quality CLI command, integrates recall-quality evidence into the evidence index and TUI /evidence dashboard, and updates the local certification script to validate these diagnostics. Feedback on the implementation suggests simplifying a redundant option check in the query evaluation logic where is_some_and is used alongside unwrap_or.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +179 to +186
if max_expected_rank.is_some_and(|expected_rank| item.rank > expected_rank) {
status = RecallQualityQueryStatus::NeedsReview;
notes.push(format!(
"expected memory {expected_top_id} returned at rank {} which is worse than allowed rank {}",
item.rank,
max_expected_rank.unwrap_or(item.rank)
));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using max_expected_rank.is_some_and(...) followed by max_expected_rank.unwrap_or(...) is redundant since we already know max_expected_rank is Some inside the block. We can simplify this by using if let Some(expected_rank) = max_expected_rank to bind the value directly, which avoids the fallback unwrap and improves readability.

                if let Some(expected_rank) = max_expected_rank {
                    if item.rank > expected_rank {
                        status = RecallQualityQueryStatus::NeedsReview;
                        notes.push(format!(
                            "expected memory {expected_top_id} returned at rank {} which is worse than allowed rank {expected_rank}",
                            item.rank
                        ));
                    }
                }

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
putComment timed out

@TerminallyLazy TerminallyLazy merged commit 4f6884e into main Jul 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant